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IMAGE READING DEVICE 

BACKGROUND OF THE INVENTION 
1 . Field of the Invention 
5 The present invention relates to an image reading device 

in which image data, having a multitude of pixel data, is 
obtained by an imaging device, and after a reading operation of 
the pixel data from the imaging device, some of the pixel data 
are thinned out . 

10 2. Description of the Related Art 

Conventionally, there is known an image reading device, 
such as a digital camera, in which image data is obtained by an 
imaging device such as a CCD, and a color image, corresponding 
to the image data, can be recorded in a recording medium. The 

15 CCD is provided with an on-chip color filter, in which color 
filter elements of the three primary colors, for example, are 
provided on photo-diodes of the CCD. Namely, a pixel data 
generated by each of the photo-diodes corresponds to one of the 
primary colors, and thus, the pixel data does not indicate the 

20 other primary colors. Thus, an interpolation process is 
performed to obtain pixel data of the other primary colors. 

A number of pixel data, recorded in the recording medium, 
is determined depending upon a recording mode, and is usually 
much less than that of the photo-diodes . Therefore, in a 

25 conventional device, after the interpolation process, some of 



pixel data are thinned out to reduce the number of pixel data in 
conformity with the number of pixel data to be recorded in the 
recording medium. 

Thus, in the conventional device, three image frame's 
worth of color image data are generated because of the 
interpolation process, and the thinning process is then applied 
to the color image data. Namely, in the interpolation process 
and the thinning process, a large number of pixel data are 
processed, and therefore, it takes a long time to obtain the 
pixel data to be recorded . 
SUMMARY OF THE INVENTION 

Therefore, an object of the present invention is to 
provide an image reading device by which the interpolation 
process and the thinning process are carried out in a reduced 
time, in comparison with a conventional device. 

According to the present invention, there is provided an 
image reading device comprising an imaging 4 device, a reading 
processor, a thinning processor and an interpolation 
processor . 

The imaging device has photo-diodes and color filters 
provided on the imaging device, the color filter having color 
filter elements of a plurality of colors. The photo-diodes 
generate an original image data containing pixel data, each of 
which corresponds to one of the colors which are arranged in a 
predetermined distribution. The reading processor reads the 





pixel data from the imaging device. The thinning processor thins 
out some of the pixel data to generate a thinned image data, 
colors of which are arranged in the predetermined distribution. 
The interpolation processor performs an interpolation process on 
5 the thinned image data to generate an interpolated image data for 
each of the colors . 

Further, according to the present invention, there is 
provided an image reading device in which pixel data of a first 
image, formed on an imaging device having an on-chip color filter 

10 of a plurality of colors, are point-sequentially read from said 
imaging device and are subjected to an interpolation process to 
generate components of said plurality of colors for each of said 
pixel data to obtain a second image, the image reading device 
comprising a thinning processor that thins out some of the pixel 

15 data before the pixel data are subjected to the interpolation 
process, so that said second image is composed of a smaller 
number of pixels than the first image. 
BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be better understood from the 

20 description of the preferred embodiments of the invention set 
forth below, together with the accompanying drawings, in 
which : 

Fig. 1 is a perspective view showing a digital camera 
which is an image reading device to which an embodiment of the 
25 present invention is applied; 



Fig. 2 is a block diagram showing electric circuits 
provided in the digital camera; 

Fig. 3 is a view showing an arrangement of an on-chip 
color filter provided on an imaging device, or an arrangement of 
color components of original image data read by the imaging 
de vi ce ; 

Fig. 4 is a view showing thinned image data, which have 
been subjected to a thinning process using a reduction ratio of 
(1/3 x 1/3) time; 

Fig. 5 is a view showing thinned image data, which have 
been subjected to a thinning process using a reduction ratio of 
(1/5 x 1/5) time; 

Fig. 6 is a view showing interpolated image data of the 
R component, which have been subjected to an interpolation 
process after the thinning process using the reduction ratio of 
(1/3 x 1/3) time; 

Fig. 7 is a view showing interpolated image data of the 
R component, which have been subjected to an interpolation 
process after the thinning process using the reduction ratio of 
(1/5 x 1/5) time; and 

Fig. 8 is a flowchart of a recording routine, by which an 
image is indicated by a liquid crystal display and image data are 
recorded in a memory card. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention will be described below with 





reference to embodiments shown in the drawings . 

Fig. 1 shows a digital camera (i.e., an image reading 
device) 10 to which an embodiment of the present invention is 
applied. By the digital camera 10, a photographed color image 
5 can be recorded in a memory card (not shown) mounted in the 
camera 10. The color image is reduced with a predetermined 
reduction ratio, so that the reduced color image can be indicated 
by a liquid crystal display (LCD) 11. 

The digital camera 10 is provided with a lens barrel 13 
10 on a front surface 10a of the camera 10 7 and a photographing 
optical system 12 is held in the lens barrel 13. On a rear 
surface 10c of the camera 10, a view-finder (not shown) is 
provided. 

The LCD 11 is provided on an upper surface 10b of the 
15 camera 10. A release button 16 is provided on the upper surface 
10b. An operation panel 15 and an indication panel 17 are also 
disposed on the upper surface 10b. The operation panel 15 is 
provided with various kinds of operation buttons including a 
reduction ratio setting button, by which the reduction ratio of 
20 the color image is set, and a cancel button, by which an 
operation of the digital camera 10 is canceled. The reduction 
ratio setting button is constructed in such a manner that the 
reduction ratio is changed to one time, (1/3 x 1/3) time and (1/5 
x 1/5) time, in this order, with every depressing operation of 
25 the button (i.e., whenever the button is depressed). The 




indication panel 17 is a liquid crystal display, for example, in 
which numerals and a predetermined figure signifying the 
reduction ratio can be indicated. 

With reference to Fig. 2, an electric construction of the 
5 digital camera 10 will be described below. 

The operation panel 15, the release button 16 and the 
indication panel 17 are connected to a control unit 40 having a 
micro-computer. Operations of the digital camera 10 are 
Q performed by depressing various kinds of operation buttons 

+;« 10 provided in the operation panel 15 or the release button 16, and 

are controlled by the control unit 40. 
'■Zl An imaging device (CCD) 30 is mounted in the digital 

camera 10. Between the photographing optical system 12 and the 
CCD 30, a quick return mirror 21 and a shutter 22 are provided, 
?S 15 the shutter 22 being disposed behind the quick return mirror 21. 
~~ A view-finder optical system 23 is positioned above the quick 

return mirror 21. 

The CCD 30 is an area sensor having a light receiving 
surface on which a multitude of photo-diodes are arranged, on 
20 which an on-chip color filter of the Bayer arrangement is 
provided. Namely, the color filter has color filter elements of 
the three primary colors, i.e., red (R) , green (G) and blue (B) , 
and thus, the photo-diodes generate an original image data 
comprising analog pixel data, each of which corresponds to one 
25 of the three primary colors . 




The quick return mirror 21 can be rotated between an 
inclined state shown in a solid line and a horizontal state shown 
in a broken line. The quick return mirror 21 is usually 
positioned in the inclined state, so that light, passing through 
5 the photographing optical system 12, is led to the view-finder 
optical system 23. In this state, the shutter 22 is closed to 
block an optical path to the CCD 30. Conversely, when the 
release button 16 is depressed to perform a photographing 
operation, the quick return mirror 21 is rotated upward under 

10 control of a mirror drive circuit 24, and is set to the 
horizontal state. While the mirror 21 is positioned at the 
horizontal state, the shutter 22 is opened under control of a 
shutter drive circuit 25, and thus, light passing through the 
photographing optical system 12 enters a light receiving surface 

15 of the CCD 30. 

A timing generation circuit 41 is connected to the 
control unit 40, so that a timing signal is generated by the 
timing generation circuit 41 under control of the control unit 
40. Based on the timing signal, a CCD drive circuit 32, a 

20 correlated double sampling circuit (CDS) 42 and an A/D converter 
43 are driven. 

When the release button 16 is depressed, a photographing 
operation is performed. Namely, the CCD 30 is controlled by the 
CCD drive circuit 32, so that one image's worth of analog pixel 
25 data of R, G and B components are point-sequentially read from 




the CCD 30, and reset noises are removed from the pixel data. 
Then, the analog pixel data are converted to digital pixel data 
by the A/D converter 43. The digital pixel data are stored in 
an image memory 45 through a digital signal processing circuit 
5 (DSP) 44. 

The digital pixel data are read from the image memory 45, 
and are subjected to a thinning process, by which a color image 
is reduced, in the DSP 44 to obtain thinned image data. The 
thinning process is carried out in accordance with a reduction 

10 ratio which is set by the reduction ratio setting button provided 
in the operation panel 15. After the thinning process, the 
thinned image data are subjected to an interpolation process, so 
that interpolated image data of R, G and B components are 
generated in accordance with a color arrangement of the LCD 11. 

15 The digital pixel data, subjected to the interpolation process, 
are subjected to predetermined processes including a color 
correction and a gamma correction, and then stored in a memory 
(not shown) provided in an LCD indication circuit 50. The 
digital pixel data, stored in the memory, are read therefrom 

20 under control of the LCD indication circuit 50, so that a color 
image is indicated by the LCD 11 based on the digital pixel data, 
i.e., the interpolated image data. 

When the image indicated by the LCD 11 is verified by the 
operator, and the release button 16 is again depressed, an image 

25 recording operation is performed, which is described below. The 



digital pixel data, which are stored in the image memory 45, have 
not been subjected to a thinning process and an interpolation 
process. When the digital pixel data are read from the image 
memory 45, the digital pixel data, are subjected to the thinning 
5 process by the DSP 44, in accordance with a reduction ratio set 
by the operation panel 15 to obtain thinned image data. Then, 
the thinned image data are subjected to the interpolation 
process, so that interpolated image data of R, G and B components 
are generated for each of the pixels of the image. The 

10 interpolated image data are then converted to luminance data and 
color differential data by a memory card controller 62 . The 
luminance data and the color differential data are then subjected 
to a predetermined process including a compression process, and 
are then stored in a memory card 60 through a memory card slot 

15 61. 

Note that the reduction ratio is usually set to 1, and 
when the image is not reduced, the digital pixel data, read from 
the image memory 45, are not subjected to the thinning process, 
but are subjected to the interpolation process. 

20 With reference to Fig. 3, an output operation of pixel 

data from the CCD 30 is described below. On the light receiving 
surface of the CCD 30, an on-chip color filter of the Bayer 
arrangement is provided, in which red filter elements, green 
filter elements and blue filter elements are arranged as shown 

25 in Fig. 3. In the drawing, U R", "G" and "B" signify the colors 
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of the filter elements, and the suffixes "m" and "n" signify 
positions of the color filter elements. 

In a row L^, R filter elements and G filter elements are 
alternately arranged, and in a row L^, G filter elements and B 
5 filter elements are alternately arranged. In rows below the row 
L^, the color filter element arrangement of the row 1^ and the 
color filter element arrangement of the row I^ +1 are alternately 
repeated. In other words, rectangular R, G and B filter elements 
are repeatedly arranged in a horizontal direction H and in a 

10 vertical direction V, with a minimum unit of a 2 x 2 matrix M. 
In the matrix M, two green filter elements are positioned on a 
diagonal line, and a red filter element and a blue filter element 
are positioned on another diagonal line. 

Each of the color filter elements corresponds to each of 

15 the photo-diodes provided on the light receiving surface of the 
CCD 30. Therefore, by each of the photo-diodes, a pixel data, 
corresponding to the color of the filter element provided on the 
photo-diode, is sensed. Pixel data sensed by the photo-diodes 
are point-sequentially output from the CCD 30. In this output 

20 operation, after the pixel data corresponding to the row 1^ are 
output, the pixel data corresponding to the row are output. 

Thus, the pixel data are output point by point in each successive 
row. Therefore, an order, in which each of the pixel data of R, 
G and B is output, is the same as an order in which each of the 

25 color filter elements is arranged as shown in Fig. 3, and in an 

10 




image formed by pixel data output from the CCD 30, each of the 
R, G and B components is repeatedly arranged with the unit of 2 
x 2 matrix M. In the following description, since the color 
arrangement of the image is the same as that of the color filter 
5 of the Bayer arrangement, it is supposed that Fig. 3 indicates 
the image, and the suffixes M m" and "n" indicate the output order 
of the pixel data, i.e., positions of pixel data in the image. 

The original image data output from the CCD 30 are 
^3 subjected to a predetermined process as described above, and are 

;tj 10 stored in the image memory 45 as digital pixel data. Addresses 

hi of the image memory 45, in which the digital pixel data of R, G 

Ml 

^ and B components are stored, correspond to the positions of the 

original image data output from the CCD 30. The digital pixel 
i^i data are read from the image memory 45, and are then subjected 

■H 15 to a thinning process so that the image is reduced, as will be 

~ described below with reference to Figs. 3 through 5. 

The thinning process is performed in such a manner that 
an arrangement of R, G and B components of the thinned image 
data, remains identical with that of the digital pixel data which 
20 are not subjected the thinning process, so that, after the 
thinning process, an interpolation process can be performed in 
a conventional method. Therefore, when the color components of 
the original image data, output from the CCD 30, are arranged in 
such a manner that a (m x m) matrix, formed by said plurality of 
25 colors, is repeated, the image can be reduced with a reduction 
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ratio S indicated by a formula (1) . 

S= (l/(mx (n-l)+l) x <l/(mx (n-l)+l) ...(1) 
wherein "n" is an arbitrary positive integer greater than 1. In 
the thinning process, (m x (n-1) ) number of pixel data are 
5 thinned out for every (m x (n-l)+l) number of pixel data in a 
horizontal direction H and a vertical direction V of the image, 
in accordance with the reduction ratio indicated by the formula 
(1) . 

In the embodiment, since "m" is 2, the image can be 

10 reduced with the reduction ratio S of (1/3 x 1/3) time (i.e., 
n=2) and (1/5 x 1/5) time (i.e., n~3) , for example. Namely, as 
the reduction ratio, one time (i.e., without reduction), (1/3 x 
1/3) time and (1/5 x 1/5) time can be selected. 

An example, in which the thinning process is performed 

15 with the reduction ratio of (1/3 x 1/3) , is described below. In 
this case, "m" is 2 and "n" is 2 in the formula (1), and thus, 
2 pixel data are thinned out for every 3 pixel data in the 
horizontal direction H and the vertical direction V of the image. 
Namely, pixel data of R, G and B components positioned in a 

20 rectangle shown by broken lines P3 in Fig. 3. In other words, 
only pixel data (R^n, G^ fn+3 , G ta¥2in , B^^, for example) which are 
not enclosed by the broken lines P3 are read from the image 
memory 45 so that the thinning process is carried out. If a 
reduced image is formed by the pixel data, which have been 

25 subjected to the thinning process, the arrangement of R, G and 

12 
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B components of the reduced image is , as shown in Fig. 4 , the 
same as that of the image, formed by the digital pixel data 
without thinning. Namely, in the reduced image, the color 
components are arranged in such a manner that a (2x2) matrix 
5 M is repeated. 

An example, in which the thinning process is performed 
with the reduction ratio of (1/5 x 1/5) , is described below. In 
this case, "m" is 2 and w n" is 3 in the formula (1) , and thus, 
4 pixel data are thinned out for every 5 pixel data in the 
10 horizontal direction H and the vertical direction V of the image, 
g Namely, pixel data of R, G and B components positioned in a 

'(J, rectangle shown by chained lines P5 in Fig. 3. In other words, 

only pixel data (R^n, G^, B+S / <3ws,»# &**s,n+s, for example) which are 
i*& t not enclosed by the chained lines P5 are read from the image 

r. - 3 

Q 15 memory 45 so that the thinning process is carried out. If a 
i'Q reduced image is formed by the pixel data, which have been 

subjected to the thinning process, the arrangement of R, G and 
B components of the reduced image is, as shown in Fig. 5, the 
same as that of the image, formed by the pixel data without 
20 thinning. Namely, in the reduced image, the color components are 
arranged in such a manner that a (2x2) matrix M is repeated. 

Fig. 6 shows pixel data of the R component, which have 
been subjected to an interpolation process after the thinning 
process using the reduction ratio of (1/3 x 1/3) time, and Fig. 
25 7 shows pixel data of the R component, which have been subjected 

13 



to an interpolation process after the thinning process using the 
reduction ratio of (1/5x1/5) time. In Figs. 6 and 7 , the pixel 
data, to which a mark is attached, is obtained by the 

interpolation process, in which the marked pixel data is 
calculated using pixel data positioned around the marked pixel 
data, and which is the same process as in a conventional device, 
since the positional relationship of R, G and B components in the 
reduced image is identical with that of the image, which is 
formed by pixel data without thinning. Namely, the marked pixel 
data is obtained by using adjacent pixel data, in such a manner 
that the adjacent pixel data are each multiplied by a weight 
coefficient, the resultant values are then totaled together, and 
then divided by the number of the adjacent pixel data, to obtain 
the marked pixel data. 

The interpolation process is performed for each of the R, 
G and B components, and thus, the pixel data of the reduced image 
composed of a single color component are obtained for each of the 
R, G and B components. The number of pixel data, which have been 
subjected to the thinning process, is less than that of pixel 
data, not subjected to the thinning process. Therefore, since 
the interpolation process is carried out for a reduced number of 
pixel data, the process time of the interpolation process is 
reduced in comparison with a conventional device. 

Fig. 8 shows a flowchart of a indication/recording 
routine, which is executed in the control unit 40 so that an 

14 



image is indicated by the LCD and image data are recorded in the 
memory card 60. The image is reduced with a reduction ratio 
which is set by the reduction ratio setting button provided in 
the operation panel 15 . 

In Step 110, it is determined whether the release button 
16 is depressed. Step 110 is executed until it is determined the 
release button 16 has been depressed, and when the release button 
16 is depressed, i.e. , when a photographing operation is started, 
the process goes to Step 120. 

In Step 120, the CCD 30 is activated, so that pixel data 
of R, G and B components of an image to be photographed are 
sensed by the CCD 30. The pixel data are subjected to a 
predetermined process, and stored in the image memory 45 as 
digital pixel data, in which addresses, in which the digital 
pixel data are stored, correspond to the arrangement of colors 
of the digital pixel data in the image, shown in Fig. 3. 

In Step 140, the reduction ratio, {1/5 x 1/5) time, for 
example, which has been set by the operation panel 15, is read, 
and positions of pixel data, which are not to be thinned out, are 
calculated so that the image is reduced in accordance with the 
reduction ratio. In Step 150, only the pixel data corresponding 
to the calculated positions are read from the image memory 45, 
by which the thinning process is completed. In Step 155, an 
interpolation process is performed to the thinned image data read 
in Step 150, so that interpolated image data of R, G and B 

15 




components are generated for each pixel of" the reduced image, 
respectively, and are stored in a memory provided in the LCD 
indication circuit 50. Due to Steps 150 and 155, the pixel data 
for indication of the image by the LCD 11 are generated. 
5 In Step 160, the interpolated image data of R, G and B 

components are read from the memory of the LCD indication circuit 
50, and based on the interpolated image data, the reduced image 
is indicated by the LCD 11. 

A loop formed by Steps 170 through 190 is executed until 
10 either one of the reduction ration setting button and the cancel 
button, which are provided in the operation panel 15, and the 
release button 16 is depressed, or the process does not 
proceed . 

In Step 170, it is determined whether the reduction ratio 
15 has been changed. The user can check the reduced image indicated 
by the LCD 11, and can depress the reduction ratio setting button 
of the operation panel 15 to change the reduction ratio. When 
it is determined in Step 170 that the reduction ratio has been 
changed, the process goes back to Step 140, so that the image is 
20 indicated by the LCD 11 with the changed reduction ratio. 
Conversely, when it is determined in Step 170 that the reduction 
ratio has not been changed, the process goes to Step 180. 

In Step 180, it is determined whether the cancel button 
of the operation panel 15 has been depressed. When it is 
25 determined that the cancel button has been depressed, i.e. when 

16 




the pixel data should not be recorded in the memory card 60 , the 
indication/recording routine ends. Conversely, when it is 
determined that the cancel button has not been depressed, Step 
190 is executed, 

5 In Step 190, it is determined whether the release button 

16 has again been depressed, i.e., whether the pixel data should 
be recorded in the memory card 60. When it is determined in Step 
190 that the release button 16 has not been depressed, Step 170 
£ =3 is again executed. Conversely, when it is determined that the 

g 10 release button 16 has been depressed, Step 200 is executed, in 

fn . . 

q which only the pixel data, corresponding to the positions 

^ calculated in Step 140, are read from the image memory 45, by 

which the thinning process is completed to obtain thinned image 
; =H data. The read thinned image data are subjected to an 

Q 15 interpolation process in Step 205, so that interpolated image 
±Q data of R, G and B components are generated, as described above. 

In Step 210, the interpolated image data of the color components 
are converted to luminance signals and color differential 
signals, and are then subjected to a predetermined process such 
20 as a data compression. Then, the processed data are stored in 
the memory card 60 through the memory card controller 62, and 
thus the indication/recording routine ends. Due to Steps 200 and 
205, the pixel data to be recorded in the memory card 60 are 
generated. 

25 According to the embodiment, when an image is reduced 

17 



# 



with a predetermined reduction ratio, firstly, a thinning process 
is performed prior to an interpolation process. Therefore, the 
number of pixel data, which are subjected to the thinning 
process, is equal to that of one image. Conventionally, when an 
5 image is reduced with a predetermined reduction ratio, the 
thinning process is performed after the interpolation process. 
In this case, pixel data read from the CCD are subjected to the 
interpolation process, so that one image's worth of pixel data 
are obtained for each of the R, G and B components. Namely, 

10 three image's worth of pixel data are generated. Therefore, 
three image's worth of pixel data are subjected to the thinning 
process, and the number of pixel data, to be processed, is great. 
In comparison, in the embodiment, the number of pixel data is not 
increased being prior to the interpolation process, and only one 

15 image's worth of pixel data are subjected to the thinning 
process. Thus, a time required for the thinning process is 
reduced . 

Further, in the embodiment, since the interpolation 
process is performed after the number of pixel data is reduced 

20 by the thinning process, useless color pixel data, for pixels not 
included in the thinned image data, are not generated, and thus 
a calculation time for the interpolation process is shortened. 
In the interpolation process, one image's worth of pixel data are 
generated for each of R, G and B components. Namely, three 

25 image's worth of color pixel data are generated. However, when 
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the interpolation process is performed, the number of pixel data 
has already been reduced to that of the reduced image, and 
therefore, the number of pixel data, which are handled in an 
image indicating operation and an image recording operation, is 
5 decreased, so that a capacity of the memory provided in the LCD 
indication circuit 50 can be reduced. 

Note that, although the on-chip color filter of the Bayer 
arrangement is provided in the embodiment, an on-chip color 
filter having the other arrangement, in which an (m x m) matrix 

10 of color components is repeatedly arranged, can be utilized. 
Further, colors of the filter elements are not restricted to R, 
G and B, and two or more color filters elements can be used. 

Although the embodiments of the present invention have 
been described herein with reference to the accompanying 

15 drawings, obviously many modifications and changes may be made 
by those skilled in this art without departing from the scope of 
the invention . 

The present disclosure relates to subject matter 
contained in Japanese Patent Application No. 10-341680 (filed on 
20 December 1, 1998) which is expressly incorporated herein, by 
reference, in its entirety. 
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